1. Field of the Invention
The present invention relates to feature extraction, and more particularly to a feature extraction system which statistically analyzes a set of samples of feature vectors to calculate a feature or characteristic serving as an index at a pattern identification, with this system allowing more accurate identification as compared with a prior system. In addition, this invention relates to a face image recognition system useful for identification systems using human faces, the certification of licenses and passports, the user identification for man-machine interface and security, and the information compression for low bit rate picture communications, and more particularly to a face image recognition system which narrows the candidates for a plausible model matching with an input image down through a first-stage model selection (first model selection means) and subsequently conducts a final recognition among the selected candidates through the use of a feature extraction different from the first stage in a second-stage selection.
2. Description of the Prior Art
The pattern identification or recognition is applicable to various fields such as a face identification or recognition available for security systems. For the pattern identification, a method based upon the KL (Karhunen-Loeve) expansion has well been known as a way of extracting a feature from a set of samples of feature vectors. This method has the ability to attain, as a feature serving as an index for the pattern recognition, an orthonormal base (eigenvector or proper vector) determined on the basis of the dispersion or variance of a set of samples of feature vectors and a value (eigenvalue) representative of the dispersion of the feature vectors in each basic direction. A description will be taken hereinbelow of the basic property of the KL expansion.
The KL expansion signifies an optimal orthogonal development under the minimum mean squared error standard. More specifically, when a subspace U(M) is produced by selecting M (.ltoreq.N) in the order of decreasing eigenvalue out of the orthonormal bases obtainable on the basis of a set of samples of N-dimensional feature vectors through the KL expansion, the subspace U(M) assumes an M-dimensional partial vector space which minimizes the mean square error from the feature vectors. In addition, the subspace U(M) is also characterised as an M-dimensional subspace which maximizes the expectation of the square distance between the feature vectors projected. These contents have been described in detail by "Statistical Pattern Recognition (Second Edition) Academic Press 1990)" Chapter 9, written by Keinosuke Fukunaga.
Furthermore, for the identification being actually made using the feature obtained from the result of the KL expansion, the feature vector closest to the input feature vector is selected from the standard data through the use of the following distance. That is, the feature vector is orthogonally projected on the aforesaid subspace U(M) to measure the distance on the subspace U(M). As the distance to be used at that time there is the Euclidean distance defined in according to the following equation or formula (1): ##EQU1## or the distance expressed by the following equation (2): ##EQU2## where .lambda..sub.i represents an eigenvalue corresponding to an eigenvector e.sub.i, and .sigma..sup.2 designates a bias.
The distance expressed by the aforesaid equation (2) coincides with the maharanobis distance when the bias .sigma. assumes zero.
As a patented feature extraction technique using the KL expansion, there has been known "Face Recognition System" disclosed in U.S. Pat. No. 5,164,992. In this patent, the two components as shown in FIG. 3 extracts a feature for the pattern identification. One of the components, designated at numeral 201, serves as a face image acquisition input section 201, while the other, denoted at numeral 202, acts as a KL expansion execution section 202. The face image acquisition input section 201 obtains a face image as an assembly of density values at every pixel (n-dimensional feature vector, where n depicts the total number of pixels of the image) and conveys it to the KL expansion execution section 202. The KL expansion execution section 202 calculates a orthonormal base (basis) through the KL expansion on the basis of N face images coming from the face image acquisition input section 201 and subsequently selects M (.ltoreq.n, N) bases in the order of decreasing eigenvalue out of the calculation results to retain the eigenvalues and the coordinates of the vectors corresponding thereto.
However, although the KL expansion is an optimal orthogonal development in view of reducing the minimum mean squared error or enhancing the scattering property of data, it is a problem in the case of using for the purpose of extracting a feature for the pattern identification. More specifically, the most important point for the patter identification is how to accurately identify the confusing feature vectors close in distance to each other, and for this purpose, it is preferable to promote only the scattering property between data close in distance to each other without enhancing the mutual scattering properties of all data. For this reason, with a prior system which carries out the feature extraction directly using the KL expansion, a problem exists in that difficulty is experienced in the identification of the confusing data.
On the other hand, for a face image recognition system, image data (for example, a variable-density image with M pixels in the vertical directions and N pixels in the horizontal directions) is completely expressible with the M.multidot.N-dimensional vectors in such a manner that each pixel is treated as one independent coordinate axis and the coordinate value is expressed as the density value of the pixel (for example, 10000 dimensions when 100.times.100). Accordingly, if L (L&gt;10000) input images are linearly independent to each other, a 10000-dimensional space becomes necessary in order to express this L-image information. However, in terms of human faces, through the recent studies, there has been known the fact that almost faces were expressible with an extremely small dimensional space. This is because as compared with general images the human faces considerably resemble each other (they have eyes, a nose, mouth and others in common and these components have a similar positional relation to each other). A detailed discussion about these matters has been made, for example, in the document "Application of the Karhunen-Loeve procedure for the characterization of human faces" (IEEE Trans. on Pattern Analysis and Machine Intelligence. vol 12, no. 1 1990), written by M. Lirby and L. Sirovich.
The KL expansion system, well-known as a recognition method of face images, takes notice of this nature of the general face images to extract a feature from the face image through the KL expansion for recognition. A detailed description is taken in the document "Face Recognition Using Eigenfaces" CVPR '91 (PROc. IEEE Conf. on Computer Vision and Pattern Recognition 1991), written by Matthew A. Turk and Alex P. Pentland. Although slightly different in input and output, the face image recognition can commonly be defined as a method which previously registers face image being models in a data base to recognize the face image of a person on the model data most similar to an input image. From this point of view, the KL expansion method is designed to approximate the an input face image I and a model image M through the linear combination of P eigenvectors Ei (i =1 . . . P) as shown by the following equation (3) taking the above description into consideration to perform the collation between the approximate data. ##EQU3## where Mrepresents an approximate value to a model face M and (M, E.sub.i) designates an inner product of vectors M and E.sub.i.
The KL method uses, as the basic vectors, the eigenvectors corresponding to P (for example, approximately 100) eigenvalues selected out of the eigenvalues of a covariance matrix obtainable from W teaching face image data in the order of decreasing magnitude. The use of this expression can reduce the expressive space of an image from the M.multidot.N (=10000) dimensions up to the approximately P (=100) dimensions while minimizing the loss of the image information contents, and hence it has hitherto been known as an effective technique for the image compression or the like, and is referred to as the KL expansion. According to this KL expansion, as a result of the extraction of a feature other than the nature on the information compression, the teaching data projected are separable most nicely, that is, distinguishable, in a space (which is referred to as an eigenface space) formed by the eigenvectors. In fact, according to the aforesaid document written by Matthew A. Turk and Alex P. Pentland, the person set of the teaching data coincides with the person set of the registered data, and a considerably excellent recognition result is attainable from the experiment in which the face of a person used for the teaching data is newly photographed to be used as an input image.
FIG. 4 is an illustration of an arrangement of a prior face image recognition system disclosed in the aforesaid document written by Matthew A. Turk and Alex P. Pentland, and referring to FIG. 4 a description will be made hereinbelow of an operation of this face image recognition system. In FIG. 4, a KL expansion unit 21 also serves as a first feature extraction means and a second feature extraction means, and implements the feature extraction from each model image existing in a model image memory 22 and stores a feature vector in a model feature vector memory 23, with these being done in the off-line processing. In the on-line, an input image, taken through a camera and retained in an object image memory 24, is transferred to the KL expansion unit 21 so that a feature vector is extracted as in the case of the model face image. Secondly, a model selection unit 25 finds out a model having a feature vector most similar to the feature vector of this input object face by checking with the contents of the model feature vector memory 23. More specifically, through the use of the basic vectors (as described before, the KL method uses, as the basic vectors, the eigenvectors corresponding to P eigenvalues (for example, approximately 100 in number) selected out of the eigenvalues of the covariance matrix obtainable from W teaching face image data in the order of decreasing magnitude) stored in an eigenspace memory (not shown), each model face vector and the input object face vector are projected onto an eigenspace (the extraction of a feature) to attain a coordinate in the eigenspace to evaluate the similarity, thus providing the output on the face image recognition. At the time, the following equation (4) is employed for the evaluation of the similarity to detect the vectors which minimizes the result of the equation (4). ##EQU4## where m.sub.l and i.sub.l represent the l.sup.th values of the feature vector and the object face feature vector.
There is a problem which arises with the prior method proposed in the aforesaid document written by Matthew A. Turk and Alex P. Pentland, however, in that the realizable system scale is small for the following reason. That is, when the KL expansion exhibits its advantages to the utmost limit in the manner of equalizing the person of the teaching data with the person of the model data, even if the number of teaching data increases, the number of effective eigenvectors obtainable undergoes restriction for the reason as mentioned above, and hence, as the number of model data (registered persons) increases, the distribution of the feature vectors in the eigenface space gets to be tight to cause the recognition ability to lower. Therefore, the number of persons registerable (or recognizable) is exposed to limitation, which makes it difficult to realize a large-scale system.